Adversarial robustness via attention transfer

Deep neural networks are known to be vulnerable adversarial attacks. The empirical analysis in our study suggests that attacks tend induce diverse network architectures shift the attention irrelevant regions. Motivated by this observation, we propose a regularization technique which enforces attentions well aligned via knowledge transfer mechanism, thereby encouraging robustness. Resultant model exhibits unprecedented robustness, securing 63.81% accuracy where prior art is 51.59% on CIFAR-10 dataset under PGD In addition, go beyond performance analytically investigate proposed method as an effective defense. Significantly flattened loss landscape can observed, demonstrating promise of for improving robustness and thus deployment security-sensitive settings.